Effects of Immersive Technology–Based Education for Undergraduate Nursing Students: Systematic Review and Meta-Analysis Using the Grading of Recommendations, Assessment, Development, and Evaluation (GRADE) Approach

Background The adoption of immersive technology in simulation-based nursing education has grown significantly, offering a solution to resource limitations and enabling safe access to clinical environments. Despite its advantages, there are still diverse reports regarding the effectiveness of immersive technology. It is crucial to verify the effectiveness of immersive technology in nursing education to inform future educational programs. Objective This systematic review aimed to identify the contents of immersive technology–based education for undergraduate nursing students and evaluate the effectiveness of immersive technology compared to traditional teaching methods. Methods A literature search was performed using 4 databases: PubMed, CINAHL, Embase, and Web of Science; the latest search was completed on January 19, 2023. The inclusion criteria were as follows: participants were undergraduate nursing students; studies were published in Korean or English; designs included randomized controlled trials (RCTs) or nonrandomized studies; and interventions involved virtual reality (VR), augmented reality (AR), mixed reality, or extended reality. Quality assessment was conducted using Cochrane Risk-of-Bias Tool version 2 for RCTs and the Risk-of-Bias Assessment Tool for Nonrandomized Studies. The main outcomes of the included studies were classified according to the New World Kirkpatrick Model (NWKM), ranging from level 1 (reaction) to level 4 (results). Meta-analysis was conducted using RevMan 5.4 software, and subgroup analysis was conducted due to heterogeneity of the results of the meta-analysis. The Grading of Recommendations, Assessment, Development, and Evaluation approach was adopted for assessing certainty and synthesizing results of the relevant literature. Results A total of 23 studies were included, with participant numbers ranging from 33 to 289. Of these, 19 (82.6%) studies adopted VR to simulate various nursing scenarios, including disaster training, resuscitation, health assessments, and home health care; 4 (17.4%) studies used AR technologies; and 15 (65.2%) studies involved virtual patients in their scenarios. Based on the NWKM, the main outcome variables were classified as level 1 (usability and satisfaction), level 2 (knowledge, motivation, confidence, performance, attitude, and self-efficacy), and level 3 (clinical reasoning); level 4 outcomes were not found in the selected studies. Results of the subgroup analysis showed that immersive technology–based nursing education is more effective than traditional education in knowledge attainment (standard mean difference [SMD]=0.59, 95% CI 0.28-0.90, P<.001, I2=49%). Additionally, there were significant difference differences between the experimental and control group in confidence (SMD=0.70, 95% CI 0.05-1.35, P=.03, I2=82%) and self-efficacy (SMD=0.86, 95% CI 0.42-1.30, P<.001, I2=63%). Conclusions These findings support the effectiveness of immersive technology–based education for undergraduate nursing students, despite heterogeneity in methods and interventions. We suggest that long-term cohort studies be conducted to evaluate the effects of immersive technology–based nursing education on NWKM level 4.


Information sources
6 Specify all databases, registers, websites, organisations, reference lists and other sources searched or consulted to identify studies.Specify the date when each source was last searched or consulted.

P3
Search strategy 7 Present the full search strategies for all databases, registers and websites, including any filters and limits used.P3 Selection process 8 Specify the methods used to decide whether a study met the inclusion criteria of the review, including how many reviewers screened each record and each report retrieved, whether they worked independently, and if applicable, details of automation tools used in the process.

Data collection process
9 Specify the methods used to collect data from reports, including how many reviewers collected data from each report, whether they worked independently, any processes for obtaining or confirming data from study investigators, and if applicable, details of automation tools used in the process.

P4
Data items 10a List and define all outcomes for which data were sought.Specify whether all results that were compatible with each outcome domain in each study were sought (e.g. for all measures, time points, analyses), and if not, the methods used to decide which results to collect.

P4
10b List and define all other variables for which data were sought (e.g.participant and intervention characteristics, funding sources).Describe any assumptions made about any missing or unclear information.

NA Study risk of bias assessment
11 Specify the methods used to assess risk of bias in the included studies, including details of the tool(s) used, how many reviewers assessed each study and whether they worked independently, and if applicable, details of automation tools used in the process.

P4
Effect measures 12 Specify for each outcome the effect measure(s) (e.g.risk ratio, mean difference) used in the synthesis or presentation of results.P4

Synthesis methods
13a Describe the processes used to decide which studies were eligible for each synthesis (e.g.tabulating the study intervention characteristics and comparing against the planned groups for each synthesis (item #5)).

P4
13b Describe any methods required to prepare the data for presentation or synthesis, such as handling of missing summary statistics, or data conversions.

P4
13c Describe any methods used to tabulate or visually display results of individual studies and syntheses.P4 13d Describe any methods used to synthesize results and provide a rationale for the choice(s).If meta-analysis was performed, describe the model(s), method(s) to identify the presence and extent of statistical heterogeneity, and software package(s) used.

P4
13e Describe any methods used to explore possible causes of heterogeneity among study results (e.g.subgroup analysis, meta-regression).NA 13f Describe any sensitivity analyses conducted to assess robustness of the synthesized results.P4

Ite m # Checklist item
Location where item is reported Certainty assessment 15 Describe any methods used to assess certainty (or confidence) in the body of evidence for an outcome.P4

Study selection 16a
Describe the results of the search and selection process, from the number of records identified in the search to the number of studies included in the review, ideally using a flow diagram.

P5-6
16b Cite studies that might appear to meet the inclusion criteria, but which were excluded, and explain why they were excluded.P5-6

Study characteristics
17 Cite each included study and present its characteristics.P5,7 Risk of bias in studies 18 Present assessments of risk of bias for each included study.P8,12-13

Results of individual studies
19 For all outcomes, present, for each study: (a) summary statistics for each group (where appropriate) and (b) an effect estimate and its precision (e.g.confidence/credible interval), ideally using structured tables or plots.

P9-11
Results of syntheses 20a For each synthesis, briefly summarise the characteristics and risk of bias among contributing studies.P14-15 20b Present results of all statistical syntheses conducted.If meta-analysis was done, present for each the summary estimate and its precision (e.g.confidence/credible interval) and measures of statistical heterogeneity.If comparing groups, describe the direction of the effect.PRISMA 2020 Checklist For more information, visit: http://www.prisma-statement.org/ of all investigations of possible causes of heterogeneity among study results.P17 20d Present results of all sensitivity analyses conducted to assess the robustness of the synthesized results.P14-15 Reporting biases 21 Present assessments of risk of bias due to missing results (arising from reporting biases) for each synthesis assessed. of certainty (or confidence) in the body of evidence for each outcome assessed.P14-15 DISCUSSION Discussion 23a Provide a general interpretation of the results in the context of other evidence.P16-18 23b Discuss any limitations of the evidence included in the review.P17 23c Discuss any limitations of the review processes used.P17 23d Discuss implications of the results for practice, policy, and future research.information for the review, including register name and registration number, or state that the review was not registered.P3 24b Indicate where the review protocol can be accessed, or state that a protocol was not prepared.NA 24c Describe and explain any amendments to information provided at registration or in the protocol.NA Support 25 Describe sources of financial or non-financial support for the review, and the role of the funders or sponsors in the review.P18 Competing interests 26 Declare any competing interests of review authors.P18 Availability of data, code and other materials 27 Report which of the following are publicly available and where they can be found: template data collection forms; data extracted from included studies; data used for all analyses; analytic code; any other materials used in the review.NA From: Page MJ, McKenzie JE, Bossuyt PM, Boutron I, Hoffmann TC, Mulrow CD, et al.The PRISMA 2020 statement: an updated guideline for reporting systematic reviews.BMJ 2021;372:n71.doi: 10.1136/bmj.n71